The ESCAPE project: Energy-efficient Scalable Algorithms for Weather Prediction at Exascale

Abstract. In the simulation of complex multi-scale flows arising in weather and climate modelling, one of the biggest challenges is to satisfy strict service requirements in terms of time to solution and to satisfy budgetary constraints in terms of energy to solution, without compromising the accuracy and stability of the application. These simulations require algorithms that minimise the energy footprint along with the time required to produce a solution, maintain the physically required level of accuracy, are numerically stable, and are resilient in case of hardware failure. The European Centre for Medium-Range Weather Forecasts (ECMWF) led the ESCAPE (Energy-efficient Scalable Algorithms for Weather Prediction at Exascale) project, funded by Horizon 2020 (H2020) under the FET-HPC (Future and Emerging Technologies in High Performance Computing) initiative. The goal of ESCAPE was to develop a sustainable strategy to evolve weather and climate prediction models to next-generation computing technologies. The project partners incorporate the expertise of leading European regional forecasting consortia, university research, experienced high-performance computing centres, and hardware vendors. This paper presents an overview of the ESCAPE strategy: (i) identify domain-specific key algorithmic motifs in weather prediction and climate models (which we term Weather & Climate Dwarfs), (ii) categorise them in terms of computational and communication patterns while (iii) adapting them to different hardware architectures with alternative programming models, (iv) analyse the challenges in optimising, and (v) find alternative algorithms for the same scheme. The participating weather prediction models are the following: IFS (Integrated Forecasting System); ALARO, a combination of AROME (Application de la Recherche à l'Opérationnel à Meso-Echelle) and ALADIN (Aire Limitée Adaptation Dynamique Développement International); and COSMO–EULAG, a combination of COSMO (Consortium for Small-scale Modeling) and EULAG (Eulerian and semi-Lagrangian fluid solver). For many of the weather and climate dwarfs ESCAPE provides prototype implementations on different hardware architectures (mainly Intel Skylake CPUs, NVIDIA GPUs, Intel Xeon Phi, Optalysys optical processor) with different programming models. The spectral transform dwarf represents a detailed example of the co-design cycle of an ESCAPE dwarf. The dwarf concept has proven to be extremely useful for the rapid prototyping of alternative algorithms and their interaction with hardware; e.g. the use of a domain-specific language (DSL). Manual adaptations have led to substantial accelerations of key algorithms in numerical weather prediction (NWP) but are not a general recipe for the performance portability of complex NWP models. Existing DSLs are found to require further evolution but are promising tools for achieving the latter. Measurements of energy and time to solution suggest that a future focus needs to be on exploiting the simultaneous use of all available resources in hybrid CPU–GPU arrangements

Elemental influences in Visinets.

<p>To achieve shown behavior in Visinets, all initial values for species A and P were set as shown and the weights for all influences (arrows) were set at 0.5, except for positive self-influence (E) where the influence of A on P was set at 0.01.</p

Transient activation (tyrosine or serine/threonine phosphorylation) of EGFR and MAPK pathway components shown in Visinets using Plot function.

<p>Each single iteration equates to the smallest incremental signal progression through the network (virtual time). R2—receptor dimer; Rp—phosphorylated receptor; RShP—Receptor-Shc-P; RShGS—Receptor-Shc-Grb-Sos complex; RafPP—phosphorylated raf1; MekPP—phosphorylated Mek1/2; ErkPP—phosphorylated Erk1/2.</p

Transient activation (tyrosine phosphorylation) of IR and IRS1.

<p>The phosphorylated species shown are the sum of multiple forms of both IR and IRS1.</p

Comparative representation of binding reaction, represented here by EGFR-EGF interaction, using ODE and CMAP formalism.

<p>Translation scheme of a chemical kinetics formalism based on ordinary differential equations (A) into a CMAP representation (B), with a full one-to-one translation protocol of all ODE terms into influences (C). The reaction rate constants (k) from ODE are represented in CMAP by weights. CMAP representation allows for second order influences (e.g., k1*[R]*[EGF]) to reflect multiple causal origin of a concept’s change.</p

EGFR signaling pathway shown in working space of Visinets in Build/Modify mode and in default Basic View.

<p>Note the use of clones for selected species, for example for RP (phosphorylated EGFR), ErkPP (double phosphorylated Erk1/2), etc. that allow the placement of the same species in different locations, thus eliminating a dense network of crossing connections and producing a “cleaner” pathway representation. The linear “core” EGFR signaling pathway is shown in light green, adaptor proteins (grb2, Shc, SOS) and their complexes (for example GS which represents Grb2 and SOS complex) are shown in dark green, and protein phosphatases (PTP1B, activated form PTP1Ba and MKP), their inactive complexes after phosphorylation, and inactive phosphorylated Receptor, RShGS, Raf and Mek (iR, iRShGS, iRaf and iMek), are shown in red. This executable model is available with embedded parameter dataset at <a href="http://www.visinets.com/pathway/list" target="_blank">http://www.visinets.com/pathway/list</a> in Featured Pathways (Supporting Information: S1 Visinets Pathway: “EGFR signaling complex with MAPK and PTP1B feedback inhibition”).</p

Simplified Insulin signaling pathway, reconstructed in Visinets as described in [2].

<p>This executable model is available with embedded parameter datasets at <a href="http://www.visinets.com/pathway/list" target="_blank">http://www.visinets.com/pathway/list</a> in Featured Pathways (Supporting Information: <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0123773#sec017" target="_blank">S2</a> Visinets Pathway: “Insulin signaling in Diabetes”).</p

Causal function.

<p><b>It describes causal interactions through concepts and weights.</b> The steepness of the function is determined by parameter <b>α</b>: the larger the value of <b>α</b>, the more responsive <i>f(x)</i> is to the input.</p

Graphical principles of conversion of an enzymatic reaction from a chemical kinetics formalism based on ordinary differential equations (A) into a CMAP representation (B), with full one-to-one translation scheme of all ODE terms into influences (C).

<p>Graphical principles of conversion of an enzymatic reaction from a chemical kinetics formalism based on ordinary differential equations (A) into a CMAP representation (B), with full one-to-one translation scheme of all ODE terms into influences (C).</p